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SUMMARY 


Problem 


Computerization and automation have integrated shipboard weapons sub- 
. systems into modern combat systems, but management of the personnel responsible 
for subsystem maintenance has not been integrated. This may be a factor con- 
tributing to the less-than-optimum electronic readiness of Navy ships. 


Purpose 


The purpose of this effort was to compare the maintenance effectiveness 
achieved by ships having the experimental combat system organizational struc- 
ture with that achieved by ships having conventional organizational structures. 


Approach 


Pilot and control ships were compared on the basis of maintenance effec- 
tiveness on common subsystems as determined by the 3M Record Layout. Equip- 
ment configuration was considered in matching pilot and control ships. Insofar 
as was possible, data were taken from comparable operational periods. The 
measures of maintenance effectiveness were: (1) number of maintenance actions, 
(2) proportion of personnel-caused equipment malfunctions, (3) total days of 
downtime, (4) average days of downtime per maintenance action, (5) total man- 
hours of maintenance, and (6) average man-hours required per maintenance action. 
Separate analyses were performed for maintenance actions that were deferred for 
spare parts or outside assistance, 


Results 


Of 300 comparisons between pilot and control ships, 177 showed no signif- 
icant difference in maintenance effectiveness, 79 appeared to favor control 
ships, 32 appeared to favor pilot ships, and in 12 cases the data were insuf~- 
ficient for statistical analysis, Half of these 300 comparisons were based 
upon frequency data, the assumption being that ships with low frequency of 
repair, total downtime, and total man-hours of maintenance would have a high 
degree of electronic readiness. There is a possibility, though, that low 
frequency data represent failure to report equipment malfunctions and that this 
is not randemly distributed between pilot and control ships. If this were true, 
maintenance frequency data could have a meaning exactly opposite to that assumed. 
It was decided, therefore, to do a separate analysis of half of the 300 compari- 
sons that were free of this potential source of experimental error. Of 150 
- measures based upon the proportion of personnel-caused equipment malfunctions 

to the number of maintenance actions, and average downtime and man-hours 
required per maintenance action, 119 indicated that there was no significant 
difference in maintenance effectiveness, 13 favored control ships, 12 favored 
pilot ships, and in 6 cases the data were insufficient for statistical analysis. 
Thus, with an elimination of frequency data, the apparent differences between 
pilot and control ships disappear. 


as % 


Conclusion 


Overall, the data fail to demonstrate that improved maintenance effective- 
ness has resulted from implementation of the combat system organizational 
structure. 


Recommendation 


The present study concerns itself solely with maintenance effectiveuess 
as defined herein. The pilot organizational structure also has had other 
impacts (e.g., on management relationships, career development, and crew 
morale) that are only partially related to maintenance effectiveness and that 
merit separate consideration. All of these factors, several of which will be 
summarized in a forthcoming final report (Note 5), should be weighed before 
a decision is made concerning the effectiveness of the combat system organiza- 
tional concept. 
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INTRODUCTION 
Problem 


Traditionally, shipboard weapons subsystems such as anti-submarine, anti- 
aircraft, and electronic warfare have been somewhat independent. The introduc- 
tion of complex computers and automation has led to the integration of these 
subsystems into what is now referred to as a ship's combat system. Management 
of the personnel responsible for maintaining these subsystems, however, has 
not been integrated, and this may be a factor contributing to the less-than- 
optimum electronic readiness of Navy ships. 


Purpose 


The purpose of the present study was to compare the maintenance effective- 
ness achieved by selected pilot ships having the experimental combat system 
organizationai structure with that achieved by selected control ships having 
conventional organizational structures. 


Background 


The combat system research program was initiated by the Bureau of Naval 
Personnel with the publication of a plan (Note 1) for developing and evaluat- 
ing an improved combat systew organization. Included in the plan was a new 
concept in shipboard organization--the integration of personnel who maintain 
ship's sensors and weapons subsystems into a single Combat System Department. 
NPRDC was assigned the task of evaluating the new concept (Note 2). Evaluation 
was to be added to the regularly scheduled mission and activities of pilot ships, 
and no changes were to be made in the mission and activities of control ships. 


An initial evaluation, covering only three pilot ships, is described in 
an earlier report (Williams & Standlee, 1975). The present evaluation repli- 
cates the data gathered on these three ships and includes data from four 
additional pilot ships. 


APPROACH 


As the evaluation was to be an added feature and as ships' operating 
schedules could not be standardized, analysis of data provided by the exist~ 
ing 3M reporting system was selected as an evaluation method that would 
minimize interference with ships' day-to-day operations. 


Selection of Pilot and Control Ships 


Equipment configuration was considered in matching pilot and control ships 
(Note 3). Individual ship evaluations are not identified in this report. Col- 
lectively, however, the pilot ships were USS HALSEY (CG 23), USS JOUETT (CG 29), 
USS TRUXTON (CGN 35), USS MACDONOUGH (DDG 39), USS MAHAN (DDG 42), USS DAHLGREN 
(DDG 43), and USS PRATT (DDG 44). The control ships were USS WORDEN (CG 18), 

USS DALE (CG 19), USS TURNER (CG 20), USS ENGLAND (CG 22), USS WAINWRIGHT (CG 28), 
USS HORNE (CG 30), USS STANDLEY (CG 32), USS FARRAGUT (DDG 37), USS LUCE (DDG 38), 
USS COONTZ (DDG 40), USS DEWEY (DDG 45), and USS PREBLE (DDG 46). 


Selection of Operational Periods 


Pilot ships entered the experimental program at different times, generally 
following a period of equipment moderization. Data-gathering time spans ranged 
from 5 to 23 months. To allow stabilization of both hardware and experimental 
organizational structure, no data were gathered within the first three months 
of a ship's entering the pilot program. Pilot and control ship comparisons are 
based upon the same number of months and, when possible, on the same calendar 
months. In some instances--for more comparable data in terms of repair and 
overhaul, post-shakedown availability, and extended overseas deployment periods-- 
different calendar months were used. 


Identification of Common Subsystems 


A computer search of relevant 3M data (Note 4) indicated that there was 
Little commonality between specific equipments on board pilot and control ships. 
The decision was made, therefore, to compare maintenance performance at the 
subsystem level as determined by the first two digits of the Equipment Identifica- 
tion Code (EIC). Thus determined, all comparisons between pilot and control 
ships were based upon the maintenance of common equipment subsystems. 


Selection of Criteria 

The following criteria of maintenance effectiveness were selected from 
data collected and supplied by the Maintenance Support Office Department (MSOD) 
3M reporting system: 


1. Number of maintenance actions. 


2. Proportion of personnel-caused equipment malfunctions (“lack of know~ 
ledge or skill," and "communication problems") to number of maintenance actions. 


+ 


3. Total days of downtime (nonoperational or reduced capability status), 
if repairs were not deferred for spare parts or outside assistance. 


k . 


4. Same as item (3), except if repairs were deferred. 


5. Average days of downtime per maintenance action, if repairs were 
not deferred. 


6. Same as item (5), except if repairs were deferred. 

7. ‘Total man-hours of maintenance, if repairs were not deferred. 

8. Same as item (7), except if repairs were deferred. 

9. Average man-hours per maintenance action, if repairs were not deferred. 


10. Same as item (9), except if repairs were deferred. 


Statistical Analysis 


The comparison of maintenance effectiveness, based on common subsystems, 
is equivalent to an experimental design involving matched-pairs observations. 
The Wilcoxon Matched-Pairs Signed-Ranks Test was used for those evaluations 
based on the total and average days of downtime and on the total and average 
man-hours of maintenance. The Chi-Square Test was used for those evaluations 
based upon the number of maintenance actions, on the proportion of personnel- 
caused equipment malfunctions, and on a summary frequency distribution of 
the Wilcoxon Test results. All tests were made at the .05 level of significance. 
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RESULTS 
Number of Maintenance Actions 


The results of a comparison of pilot and control ship maintenance per- 
formance based on the number of reported maintenance actions are presented 
in Table 1, The length of the test period was the same for each pilot ship 
and its control, but varied from 5 to 23 months among the different pilot- 
control comparisons. Tests of significance are based upon the Chi-Square 
Test and are at the .05 level of confidence. 


Table 1 indicates that, in 20 comparisons, control ships reported fewer 
maintenance actions; in 4 comparisons, pilot ships reported fewer maintenance 
actions; and in 6 comparisons, the differences were not significant. 


The single fact that a majority of control ships reported fewer maintenance 
actions does not necessarily mean that electronic readiness was lower for pilot 
ships. There is a possibility that low frequency data represent failure to 
report equipment malfunctions and that this is not randomly distributed b«tween 
pilot and control ships. This might mean, then, that pilot ships were more 
diligent in reporting maintenance actions or that--possibly through better test 
procedures and higher equipment performance standards~-they located a greater 
percentage of their casualties. Thus, interpretation of the data of Table 1 
concerning electronic readiness is uncertain. 


Proportion of Personnel-caused Equipment Malfunctions 


The results of a comparison of pilot and control ship maintenance per- 
formance based on the proportion of total maintenance actions that were caused 
by ships' personnel (i.e., equipment failure caused by “lack of knowledge or 
skill" or "communication problems") are presented in Table 2. 


Table 2 indicates that, in 6 comparisons, control ships reported a lower 
proportion of personnel-caused equipment malfunctions and, in 24 comparisons, 
the differences were not significant. In terms of this criterion, control 
ships appear to have the advantage. 


Downtime and Maintenance Man-hours 


The results of a comparison of pilot and control ship maintenance per- 
formance based on days of downtime (non-operational or reduced capability 
status) and man-hours expended in making repairs are presented in Tables 3, 4, 
and 5. Table 3 compares performance based on casualties where repairs were 
not deferred for spare parts or outside assistance. Table 4 compares per- 
formance based on casualties where repairs were deferred. Table 5 summarizes 
the fairly extensive data of Tables 3 and 4. 


Table 3 indicates that, in 16 comparisons, control ships reported fewer 
non-deferred days of downtime; in 2 comparisons, pilot ships reported fewer 
days down; and in 12 comparisons, the differences were not significant. In 
15 comparisons, control ships reported fewer non-deferred man~hours of main- 
tenance; in 1 comparison, a pilot ship reported fewer man-hours of maintenance; 
and in 14 comparisons, the differences were not significant. 


Table 1 
Differences Between Pilot and Control Ships in 


Number of Maintenance Actions 


=. (itonthsy® — Dumber of Maintenance AceiCes = Bewer 
Pilot Control Pilot Control Difference Actions 
Replication of Initial Ship Data 

l 1 (11) 222 233 -11 - 

1 2 (22) 740 660 80 ¢ 

l 3 (22) 768 475 293 re 

1 4 G13) 245 382 -137 p 

l 5 (12) 220 58 162 C 

2 6 (23) 857 901 -44 - 

2 7 (23) 858 1149 -291 P 

2 8 (23) 843 460 383 C 

2 9 (19) 728 245 483 C 

3 10 (15) 744 220 524 C 

3 ll C5) 846 888 -42 - 

5 2 (15) 832 846 -14 - 

Additional Ship Data 

4 1 CET) 260 238 Oe: - 

4 2 (18) 707 395 Siz Cc 

4 3 (18) 719 311 408 C 

4 4 (14) 545 385 160 c 

4 5 (12) 423 58 365 C 
I ee See ee LA eee a SE ee ee ee ee ee 

5 1 (11) 332 236 96 Cc 

5 2 (17) 652 379 273 c 

5 3 CL) 663 279 384 C 

5 4 (12) 420 378 42 - 

5 5 (ey: 373 58 315 C 
Ne ee a ee ae SS EE ee ee ee eee 

6 l G7) 387 205 182 € 

6 2 (aA 388 197 191 C 

6 3 7 381 74 307 Gs 

6 4 (7) 382 249 133 G 

6 5 (7); 303 25 278 C 
a foe ae Oe ee DE A ee ee ee ee eee 

7 10 (5S) 27 6 21 & 

7 ll (5) 35 155 -120 P 

7 12 (5) 36 155 -119 P 


i 


Note. Significantly fewer maintenance actions in pilot ships is indicated 
by "P'", in control ships by "C'"; no significant difference is indi- 
cated by "-", Tests of significance are based on the Chi-Square Test, 
equal ratio hypothesis, .05 level of confidence. 


*pilot and control ships were not always matched on the same calendar 
months; therefore, frequency data may differ for the same number of months. 
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Table 2 


Differences Between Pilot and Control Ships in 


Personnel-caused Equipment Malfunctions 
ee RT 


Ship Number of Malfunctions oe 


Pilot Control Pilot Control Difference Proportion 


Je I eS SS eee 
SS 


Replication of Initial Ship Data 


Note. Significantly lower proportion of personnel-caused equipment mal- 
functions in pilot ships is indicated by "P", in control ships by 
"C's no significant difference is indicated by "-". Tests of signifi- 
cance are based on the Chi-Square Test, equal proportion hypothesis 
(data of Tables 1 and 2 combined), .05 level of confidence. 
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difference is indicated by "~"', 


Table 3 
Differences Between Pilot and Control Ships in 


Days of Downtime and Maintenance Man-hours (Without Deferral) 


Days of Downtime 


Average 


Replication of Initial Ship Data 


) Carts 


- 


Significantly fewer days of downtime or maintenance man-hours in pilot 
ships is indicated by "P", in control ships by "C"; no significant 
Tests of significance are based on 
the Wilcoxon Matched-Pairs Signed-Ranks Test applied to the raw data 
presented in the appendix, .05 level of confidence. 


Maintenance Man-hours 


eee € 


Table 4 
Differences Between Pilot and Control Ships in 


Days of Downtime and Maintenance Man-hours (With Deferral) 


Ee eee ee eee ee al 


Fewer Fewer 
Ship Days of Downtime Maintenance Man-hours 
Pilot Control Total Average Total Average 
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Replication of Initial Ship Data 


1 1 “ = = S 

1 2a - - P P 

1 3 ~ p p Pp 

1 4 P = P 3 

1 5 = - ie = 
cep iene ee ini ee A A LAO CIID 

2 6 P ~ = 

2 7 - = = c 

2 8 o = fe 2 

2 9 ~~ es 
ee See ee 

3 10 = m= i % 

3 18 | = = = 

3 12 P - - - 
ee a nn eta el Aan OAL EAD DLE ALATA 

Additional Ship Data 

CTD oe. eee | ee a: 5 ageneees Ra a 

4 1 C - c - 

4 2 C P C - 

4 3 C - - P 

4 4 - P C C 

4 5 G - Cc - 
a rr 

5 1 c (Ri E ti 

5 is C - - - 

5 3 Lo C - P 

S 4 P - - - 

5 5 C - G - 
ee ee Pe a nn Se 

6 1 Pp o %e 

6 2 P P P P 

6 3 - - we ra 

6 4 P 4 P - 

6 2 - - - - 
Te Ter ein a aera a 

7 10 ID ID ID ID 

7 11 D ID ID ID 

7 12 ID ID ID ID 


ie pallens ent 

Note. Significantly fewer days of downtime or maintenance man-hours in 
pilot ships is indicated by "pt in control ships by “C"; no signi- 
ficant difference is indicated by "-" and insufficient data by "ID". 
Tests of significance are based on the Wilcoxon Matched-Pairs Signed- 
Ranks Test applied to the raw data presented in the appendix, .05 
level of confidence. 
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Table 5 


Summary of Differences Between Pilot and Control Ships in 
Days of Downtime and Maintenance Man-hours 


(With and Without Deferral) 


Frequency — 
Difference Variable Pilot Control No Signifi- Total 


Ship Ship cant Dif 


Fewer Days of Downtime 
Total 9 25 23 57 
Average 6 4 47 57 


Fewer Maintenance Man-hours 
Total 7 21 29 57 
Average 6 3 48 LY i 


i  ———————— 


Note. The frequency distributions of pilot and control ships with lower 
total, lower average, and no difference in days of downtime and 
maintenance man-hours are significantly different from chance dis- 
tribution, Chi-Square Test, equal ratio hypothesis (data of Tables 
3 and 4 combined), .05 level of confidence. 


In terms of average non-deferred downtime and average non-deferred man- 
hours required per maintenance action many of the foregoing differences 
between pilot and control ships disappear. In 2 comparisons, control ships 
reported lower average days down; in 2 comparisons, pilot ships reported 
lower average days down; and in 26 comparisons, the differences were not 
significant. In 1 comparison, a control ship reported lower average man- 
hours of maintenance; in 1 comparison, a pilot ship reported lower average 
man-hours of maintenance; and in 28 comparisons, the differences were not 
significant. 


Table 4 indicates that, in 9 comparisons, control ships reported fewer 
deferred days of downtime; in 7 comparisons, pilot ships reported fewer days 
down; in 11 comparisons, the differences were not significant; and in 3 
comparisons, there were insufficient data. In 6 comparisons, control ships 
reported fewer deferred man-hours of maintenance; in 6 comparisons, pilot 
ships reported fewer man-hours of maintenance; in 15 comparisons, the dif- 
ferences were not significant; and in 3 comparisons, there were insufficient 
data, 


In terms of average deferred downtime and average deferred man-hours 
required per maintenance action, the data appear to shift slightly in favor 
of pilot ships. In 2 comparisons, control ships reported lower average days 
down; in 4 comparisons, pilot ships reported lower average days down; in 21 
comparisons, the differences were not significant; and in 3 comparisons; there 


were insufficient data. In 2 comparisons, control ships reported lower 
average man-hours of maintenance; in 5 comparisons, pilot ships reported 
lower average man-hours of maintenance; in 20 comparisons, the differences 
were not significant; and in 3 comparisons, there were insufficient data. 


In summarizing the comparison of pilot and control ship performance 
based on days of downtime and man~hours expended in making repairs, Table 5 
shows that, in the majority of cases, there is no significant difference in 
pilot and control ship performance. There is a definite trend, though, for 
control ships to report fewer total days of downtime and total man-hours of 
maintenance. There is also a slight trend for pilot ships to report lower 
average days of downtime and average man-hours required per maintenance action. 
The latter would appear to be the more valid indicator of maintenance pro- 
ficiency, as it corrects for any difference that may exist in ships actually 
reporting casualty data. 
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DISCUSSION 


The maintenance effectiveness of pilot and control ships was compared 
on the basis of data collected and supplied by the MSOD 3M reporting system. 
The measures used were: (1) number of maintenance actions, (2) proportion 
of personnel-caused equipment malfunctions, (3) total days of downtime, (4) 
average days of downtime per maintenance action, (5) total man-hours of 
maintenance, and (6) average man-hours required per maintenance action. 
Separate analyses were performed for maintenance actions that were deferred 
for spare parts or outside assistance. 


Of the 300 comparisons made between pilot and control ships, 177 indicated 
that there was no significant difference in maintenance effectiveness, 79 
appeared to favor control ships, 32 appeared to favor pilot ships, and in 
12 cases the data were insufficient for statistical analysis. 


The fact that control ships reported fewer maintenance actions (which 
would influence total days down and total man-hours of maintenance) may not 
mean that electronic readiness is lower on pilot ships. It may mean that 
pilot ships are more diligent in reporting maintenance actions or that-- 
possibly through better test procedures and higher equipment performance 
standards--they locate a greater percentage of their casualties. This 
possibility cannot be determined from the present data. However, it does 
appear to be generally accepted that Navy ships differ considerably in the 
degree to which they conform to 3M data reporting requirements, and it may 
be worthwhile to examine separately that data that is free of this potential 
source of experimental error. 


Of the 150 comparisons made between pilot and control ships that were 
based upon the proportion of personnel~caused equipment malfunctions to the 
number of maintenance actions, to the average days down, and to the average 
man-hours expended per maintenance action, 119 indicated that there was no 
significant difference in maintenance effectiveness, 13 favored control ships, 
12 favored pilot ships, and in 6 cases the data were insufficient for statistical 
analysis. Thus, by eliminating frequency data, the apparent differences be- 
tween pilot and control ships disappear. 
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CONCLUSION 


Overall, the data fail to demonstrate that improved maintenance effective- 
ness has resulted from implementation of the combat system crganizational 
structure on board pilot ships. 


RECOMMENDATION 


The present study concerns itself solely with maintenance effectiveness as 
defined herein. The pilot organizational structure also has had other impacts 
(e.g-, On management relationships, career development, and crew morale) that 
are only partially related to maintenance effectiveness and that merit separate 
consideration. All of these factors, several of which will be summarized in a 
forthcoming final report (Note 5), should be weighed before a decision is made 
concerning the effectiveness of the combat system organizational concept. 
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